On the Generalisation of Soft Margin Algorithms

Generalisation bounds depending on the margin of a classiier are a relatively recent development. They provide an explanation of the performance of state-of-the-art learning systems such as Support Vector Machines (SVM) and Adaboost. The diiculty with these bounds has been either their dependence on the minimal margin or their agnostic form. The paper presents a technique for correcting those points that fail to meet a target margin, hence creating a large margin classiier at the expense of additional functional complexity. Analysis of this technique leads to bounds that motivate the previously heuristic soft margin SVM algorithms as well as justifying the use of the quadratic loss in neural network training algorithms. The results are extended to give bounds for the probability of failing to achieve a target accuracy in regression prediction.